Fuzzy membership function based neural networks with applications to the visual servoing of robot manipulators

It is shown that there exists a nonlinear mapping which transforms image features and their changes to the desired camera motion without measuring of the relative distance between the camera and the object. This nonlinear mapping can eliminate several difficulties occurring in computing the inverse of the feature Jacobian as in the usual feature-based visual feedback control methods. Instead of analytically deriving the closed form of this mapping, a fuzzy membership function (FMF) based neural network incorporating a fuzzy-neural interpolating network is proposed to approximate the nonlinear mapping, where the structure of the FMF network is similar to that of radial basis function neural network which is known to be very effective in the function approximation. Several FMF networks are trained to be capable of tracking a moving object in the whole workspace along the line of sight. For an effective implementation of the proposed FMF network, an image feature selection process is investigated, and the required fuzzy membership functions are designed. Finally, several numerical examples are presented to show the validity of the proposed visual servoing method. I. 1NTRODUCTION ECENTLY, the visual feedback control has received R much attention particularly in the intelligent robotics field by virtue of its power of allowing a robot to manipulate and track a randomly moving object without any previous knowledge of the object’s location or motion [ I ] . Applications of visual feedback systems include the seam tracking [2], the precision part placement [3], the conveyor tracking 141, and the control of space telerobots [ 5 ] . The actual image features have been widely employed as feedback signals in visual robot systems. In the image featurebased visual feedback structure [6] is usually required the computation of the inverse of the ,feature Jacobian, which is diffijr-entia1 relationship heh>een U feature space and a camera motion space, for computing the desired camera motion based on the image feature changes of objects. However, using the inverse of the feature Jacobian cannot accommodate rather large feature variations due to its corresponding large motion errors. Thus, other auxiliary techniques may be necessary to compensate for such motion errors [71-[91. Feddema and Mitchell [7] proposed a feature-based dynamic look-and-move control structure allowing the visual feedback Manuscript received April 23, 1992: revised February 8, 1993; June 2, 1993. This work was supported in part hy (’ontrol Systems Lab. of KIST and ERC-ACI of SNU by KOSEF. The authors are with the Department of Electronics Engineering, Hanyanp University. Haengdang-dong 17. Songdong-ku. Seoul 133.79 1. Korea. IEEE Log Number 921 1876. to have variable sampling and delay times by incorporating the generation of feature-based trajectories as well as the feature Jacobian. However, the feature Jacobian must be manually generated before a task begins. In-depth knowledge of the robot kinematics and the camera modelling is not a trivial task. Hence, in [8] and [9], the model reference adaptive controller (MRAC) has been employed to minimize coupling effects in transforming the coordinate of feature space to that of the joint space. But computational requirements for the realtime parameter identification, and the sensitivity to numerical precision, sensor noise, and choice of model might make the approach infeasible as the number of system variables increases. In spite of the use of such auxiliary techniques, there are some additional drawbacks in using the feature Jacobian approach [7]-[9]. First, feature Jacobians in [7]-[9] could be the inherent minor error source when the usual pin-hole camera model is assumed. Second, the feature Jacobian approach requires estimation of the distance between the object and the camera, which is usually solved by employing depth measuring techniques such as laser range finders, binocular stereo [lo], active monocular stereo [ 111, or computed from the objects’ CAD model data [1],[7]. This makes the feature Jacobian approach too complicated to implement. And even when equipping the controller with a distance measurement device, the output tracking performance may heavily depend upon the choice of features [I], because the feature Jacobian is very sensitive to feature variations. Finally, in the worst case, the feature Jacobian might be singular, which drives the system uncontrollable. On the other hand, when we consider that living animals including human beings just see and pick up an object without precise numerical position information, the visual servoing seems to be an innate ability of living animals. Such a skill often improves by repetitive learning while growing up. Insinuated by the above observation, some researchers have attempted to develop not only methods for the robot motion control utilizing the image features, but also schemes enabling a robot to elaborate the motion for itself by repetitive trials Among them, Miller [4], [ 121 proposed a neural network based learning control system for the visual servoing, where the CMAC (Cerebeller Model Arithmetic Computer) memory was employed for the learning. In this control system, a priori knowledge of neither the robot kinematics nor the object speed [41, [121, ~ 3 1 . 1063-6706/94$04.00